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DETAILED ACTION 

1 . The Request for Continued Examination (RCE) filed on 3 May 2006 under 37 CFR 
1.53(d) based on parent Application No. 09/976,188 is acceptable and a RCE has been 
established. An action on the RCE follows. 

2. The amendments filed on 3 May 2006, submitted with the filing of the RCE have been 
received and entered. Claims 1-33 as amended are pending in the application. 

Claim Rejections - 35 USC § 112 
The following is a quotation of the first paragraph of 35 U.S.C. 1 12: 

The specification shall contain a written description of the invention, and of the manner and process of making 
and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it 
pertains, or with which it is most nearly connected, to make and use the same and shall set forth the best mode 
contemplated by the inventor of carrying out his invention. 

3. Claims 1-33 are rejected under 35 U.S.C 1 12, first paragraph, as failing to comply with 
the written description requirement. The claim(s) contains subject matter which was not 
described in the specification in such a way as to reasonably convey to one skilled in the relevant 
art that the inventor(s), at the time the application was filed, had possession of the claimed 
invention. The applicant's disclosure does not provide sufficient support the claimed subject 
matter of allowing a user to enter handwritten data while still being operable to simultaneously 
receive typed input data from a physical keyboard. The specification states that "A user may 
enter commands and information into the computer 20 through input devices such as a tablet 
(electronic digitizer) 164, a microphone 163, a keyboard 162 and pointing device 161, etc." on 
lines 10-13 of page 4, and correspondingly shown in Figure 1. However, although the 
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specification and Figure 1 show a physical keyboard 162, the disclosure does not recite or show 
the ability to simultaneously receive handwritten input and typed input from the physical 
keyboard. In other words, although the specification describes that the user can tap on a text 
field, causing a semi-transparent user interface window to be displayed for receiving handwritten 
input (page 17, line 5-page 21, line 9), the specification does not provide any disclosure of the 
text field still receiving typed input from a physical keyboard WHILE the semi-transparent user 
interface window is receiving handwritten input. Therefore, the applicant's disclosure does not 
describe the claimed limitation in such a manner that would enable one of ordinary sill in the art 
to determine that the inventors had possession of the claimed invention. 

The dependent claims are rejected for fully incorporating the deficiencies of the base 

claim. 

4. Claims 1-33 are rejected under 35 U.S.C. 1 12, first paragraph, as failing to comply with 
the enablement requirement. The claim(s) contains subject matter which was not described in 
the specification in such a way as to enable one skilled in the art to which it pertains, or with 
which it is most nearly connected, to make and/or use the invention. The applicant claims that 
typed input from a physical keyboard and handwritten data can be simultaneously received. The 
examiner is unclear on how input can be received from two different input systems (i.e. a 
keyboard and handwritten input via a stylus pen, for example) at the same time because the 
specification does not describe steps or means for allowing two inputs to be simultaneously 
received; in other words, if the user is simultaneously typing on the keyboard and handwriting 
data via a pen, how will the system handle both inputs? Will the two inputs contradict each 
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other? The disclosure does not describe any means for enabling the system to simultaneously 
handle the inputs received form the keyboard and the handwritten input system. Therefore, the 
specification fails to teach how to make and use the claimed invention without undue 
experimentation. 

The dependent claims are rejected for fully incorporating the deficiencies of the base 

claim. 



The following is a quotation of the second paragraph of 35 U.S. C. 112: 

The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the 
subject matter which the applicant regards as his invention. 

5. Claims 1-33 are rejected under 35 U.S.C. 1 12, second paragraph, as being indefinite for 
failing to particularly point out and distinctly claim the subject matter which applicant regards as 
the invention. Due to deficiencies in the disclosure, it is not apparent how the claimed subject 
matter (namely, simultaneously receiving handwritten input and typed input from a physical 
keyboard) is possible. 

6. Furthermore, claims 18-33 are indefinite for failing to particularly point out and distinctly 
claim the subject matter which applicant regards as the invention because of the use of indefinite 
language such as "may" (i.e. "data may be entered" on line 8 of claim 18 and "a user may still 
simultaneously enter data via user-typed data input" on lines 1 1-12 of claim 30). It is unclear 
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whether data is actually being entered/user is actually entering data or not, therefore, the claims 
are indefinite. 

The dependent claims are rejected for fully incorporating the deficiencies of the base 

claim. 

Claim Rejections - 35 USC §103 
The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

7. Claims 1-4, 6-10, 12-15, 17-21, 23, 25-29 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Partanen et al. U.S. Publication 2003/0001899 (hereinafter "Partanen") and 
Skinner U.S. Patent 6,661,920. 

Referring to claims 1 and 18, Partanen teaches a method and system comprising user 
interface code evaluating a program field that has focus against information indicative of 
whether the field is configured to receive text input (the interface determines whether the user 
has initiated manuscript input to any point on a touch-activated screen of a display, thereby 
causing a semi-transparent window to be opened) (Partanen: page 1, paragraph 0016); and if the 
field is configured to receive text input (if a user activated manuscript input has been received) 
(Partanen: page 1, paragraph 0016), providing a visible user input interface at a displayed 
location relative to the field such that the user input interface is operable to receive handwritten 
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data while the field is operable to receive input data (upon user initiation of manuscript input at 
point 36 on the display screen of Figure 1, a semi-transparent window is displayed relative to the 
field, allowing users to input manuscript characters, for example, the letter "I", while the field is 
operable to receive input data such as user input of particular letters and/or numbers from a 
virtual keyboard) (Partanen: page 2, paragraph 0032 and page 3, paragraph 0038); receiving 
handwritten data at the input interface (Partanen: page 2, paragraph 0022 and Figure 1); 
providing the handwritten data to a recognition engine (relaying the pattern of the handwritten 
character to the handwriting recognition software) (Partanen: page 2, paragraph 0025); and 
returning a recognition result to the program (for example, recognizing the handwritten "I" and 
outputting it as the character "I" in Figure 1) (Partanen: page 2, paragraph 0022). However, 
although Partanen teaches a virtual keyboard, Partanen fails to explicitly teach simultaneously 
receiving typed and handwritten user input. Skinner teaches an interface that allows users to 
enter handwritten data similar to that of Partanen. In addition, Skinner further teaches receiving 
typed user input and the program field being operable to simultaneously receive typed and 
handwritten user input (the handwriting and typed keyboard input systems are simultaneously 
active and capable of data entry) (Skinner: column 2, line 41-column 3, line 3). It would have 
been obvious to one of ordinary skill in the art, having the teachings of Partanen and Skinner 
before him at the time the invention was made, to modify the handwriting recognition method of 
Partanen to include the simultaneous handwriting and typed keyboard input capabilities of 
Skinner. One would have been motivated to make such a combination in order to provide a 
flexible user data entry system, giving the user an option to choose the more convenient input 
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mechanism depending on the function to be executed without the hassle of having to switch 
between the input systems. 

Referring to claims 2 and 19, Partanen, as modified, teach the visible user input interface 
is semi-transparent (Partanen: page 1, paragraphs 0015-0016). 

Referring to claim 3, Partanen, as modified, teach wherein the handwritten data received 
at the input interface is evaluated to determine whether the handwritten data corresponds to a 
gesture (deciphering whether user input is a gesture, such as user selection of virtual buttons and 
keys from a virtual keyboard, or user input of written manuscript characters that requires 
handwriting recognition) (Partanen: pages 2-3, paragraphs 0032-0040). 

Referring to claim 4, Partanen, as modified, teach wherein the handwritten data 
corresponds to a gesture, and further comprising, providing at least one pen event corresponding 
to the gesture to the program (for example, if the handwritten data is a gesture of user selection 
of a virtual button 44 with an external input device such as the pen 30 shown in Figure 2, then 
the pen selection of virtual button 44 causes an event such as deactivation of the semi-transparent 
window) (Partanen: page 3, paragraphs 0038-0040). 

Referring to claims 6 and 26, Partanen, as modified, teach providing the handwritten data 
to a recognition engine in response to detection of a submit button associated with the visible 
user interface (the semi-transparent user interface provides for the display of a virtual keyboard 
having letters, or buttons, the users can select; furthermore, user handwritten data is recognized 
after the user gesture of touching a field with the stylus to indicate handwriting and causing 
display of the semi-transparent window) (Partanen: page 1, paragraphs 0014-0016, page 2, 
paragraph 0032 and page 3, paragraphs 0035 and 0038). 
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Referring to claim 7, Partanen, as modified, teach providing the handwritten data to a 
recognition engine is performed in response to a time being achieved (displaying the handwritten 
input for a predetermined period of time after the input from the input device, upon which the 
pattern is completed and recognized by the recognition software) (Partanen: page 2, paragraph 
0024 and page 3, paragraphs 0033-0035). 

Referring to claim 8, Partanen, as modified, teach wherein providing the handwritten data 
to a recognition engine is performed in response to a gesture being detected (user handwritten 
data is recognized after the user gesture of touching a field with the stylus to indicate 
handwriting and causing display of the semi-transparent window) (Partanen: page 1, paragraphs 
0014-0016 and page 2, paragraph 0032). 

Referring to claims 9 and 20, Partanen, as modified, teach evaluating at least one window 
attribute corresponding to the field against hard-coded or retrieved information to determine 
whether the field is supported (evaluating, or determining the window attribute of whether the 
user has contacted the displayed window, or screen, with the stylus) (Partanen: page 1, paragraph 
0016 and page 2, paragraph 0032). 

Referring to claim 10, Partanen, as modified, teach accessing window class information 
(accessing window information such as determining whether user has contacted the windowed 
area with a stylus) (Partanen: page 1, paragraph 0016 and page 2, paragraphs 0027-0030 and 
0032). 

Referring to claim 12, Partanen, as modified, teach adjusting the appearance of the visible 
input window (the semi-transparent window may be moved or sized according to the preferences 
of the user) (Partanen: page 1, paragraph 0015). 
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Referring to claim 13, Partanen, as modified, teach increasing the size of the visible input 
window to enable entry of additional handwritten data (increasing the size of the semi- 
transparent window) (Partanen: page 2, paragraph 003 1). 

Referring to claim 14, Partanen, as modified, teach erasing the visible input window 
(selection of the virtual button 44 will cause the semi-transparent window to de-activate and 
disappear) (Partanen: page 3, paragraph 0039). 

Referring to claim 15, Partanen, as modified, teach the visible input window is erased in 
response to a close request (selection of the virtual button 44 will cause the semi-transparent 
window to de-activate and disappear) (Partanen: page 3, paragraph 0039). 

Referring to claim 17, Partanen, as modified, teach wherein the visible input window is 
erased in response to a gesture being detected (upon user gesture of selection of virtual button 
44, the semi-transparent input window is deactivated and disappears) (Partanen: page 3, 
paragraph 0039). 

Referring to claim 21, Partanen, as modified, teach wherein the entered data comprises 
handwritten data (user input of handwritten characters) (Partanen: page 2, paragraph 0022 and 
Figure 1), and further comprising a gesture detection engine that evaluates the handwritten data 
to determine whether the handwritten data corresponds to a gesture (deciphering whether user 
input is a gesture, such as user selection of virtual buttons and keys from a virtual keyboard, or 
user input of written manuscript characters that requires handwriting recognition) (Partanen: 
pages 2-3, paragraphs 0032-0040), and if so, to provide at least one event to the program (for 
example, if the handwritten data is a gesture of user selection of a virtual button 44 with an 
external input device such as the pen 30 shown in Figure 2, then the pen selection of virtual 
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button 44 causes an event such as deactivation of the semi-transparent window) (Partanen: page 
3, paragraphs 0038-0040). 

Referring to claim 23, Partanen, as modified, teach wherein the entered data comprises 
handwritten data (user input of handwritten characters) (Partanen: page 2, paragraph 0022 and 
Figure 1), and further comprising a rulebase that determines an appearance of the visible input 
area including a displayed size thereof (pre-selected preferences that may be established such as 
the display size of the semi-transparent window) (Partanen: page 2, paragraphs 0027-003 1). 

Referring to claim 25, Partanen, as modified, teach wherein the visible input area has at 
least one button associated therewith for receiving a command (the semi-transparent window has 
a plurality of buttons that can be selected by the user) (Partanen: page 3, paragraphs 0038-0039 
and further shown in Figure 1). 

Referring to claim 27, Partanen, as modified, teach wherein the user input interface code 
provides the recognition result to the program in a message queue associated with the program 
(as each handwriting character is written, the display signal for displaying the recognized 
handwriting is generated and each characters is displayed on the screen in a queue, i.e. one at a 
time) (Partanen: page 2, paragraphs 0024-0025 and further shown in Figure 1). 

Referring to claim 28, Partanen, as modified, teach wherein the drawing of the visible 
input area positions the visible input area relative to the field based on the information received 
from the field typing engine (the semi-transparent window is displayed at a particular position 
and in a particular size corresponding to user pre-selected preferences) (Partanen: page 2, 
paragraphs 0027-0031). 
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Referring to claim 29, Partanen, as modified, teach wherein the drawing of the visible 
input area sizes the visible input area based on the information received from the field typing 
engine (the semi-transparent window is displayed at a particular position and in a particular size 
corresponding to user pre-selected preferences) (Partanen: page 2, paragraphs 0027-0031). 

8. Claims 5, 16 and 22 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Partanen et al. U.S. Publication 2003/0001899 (hereinafter "Partanen") and Skinner U.S. Patent 
6,661,920, as applied to claims 1 and 18 above, and Frink et al. U.S. Patent 5,956,423 
(hereinafter "Frink"). 

Referring to claims 5 and 22, Partanen and Skinner teach all of the limitations as applied 
to claims 1 and 18 above. Specifically, Partanen and Skinner teach a semi-transparent user 
interface and user input gestures (Partanen: page 1, paragraph 0016, page 3, paragraphs 0038- 
0040 and further shown in Figure 2). However, Partanen and Skinner fail to explicitly teach the 
gesture comprising user input directed to an area of the program that is visible through the user 
interface. Frink teaches an interface for inputting and recognizing handwritten data (Frink: 
column 2, line 37-column 3, line 4) similar to that of Partanen and Skinner. In addition, Frink 
further teaches user inputting gestures directed to an area of the program that is visible through 
the interface (Frink: column 2, line 37-column 3, line 4 and column 3, lines 52-64). It would 
have been obvious to one of ordinary skill in the art, having the teachings of Partanen, Skinner 
and Frink before him at the time the invention was made, to modify the handwriting input 
interface comprising gesture detection of Partanen and Skinner to include the input of gestures 
taught by Frink. One would have been motivated to make such a combination in order to allow 
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users to easily edit the documents, reducing the confusion of mixing up editing commands and 
data input by the user. 

Referring to claim 16, Partanen, as modified, teach erasing the input window in response 
to a time being achieved (sending the handwritten data to the recognition engine, and therefore 
erasing the input, when the user stops writing for a period of time) (Frink: column 2, lines 44-63 
and Figures 2A, 2B and 2C). 

9. Claim 1 1 is rejected under 35 U.S.C. 103(a) as being unpatentable over Partanen et al. 
U.S. Publication 2003/0001899 (hereinafter "Partanen") and Skinner U.S. Patent 6,661,920, as 
applied to claim 1 above, and Gilai et al. U.S. Patent 6,018,736 (hereinafter "Gilai"). 

Referring to claim 1 1, Partanen and Skinner teach all of the limitations as applied to 
claim 1 above. Specifically, Partanen and Skinner teach obtaining information indicative of 
whether the field is configured to receive text input (determining whether the user has initiated 
manuscript input to any point on a touch-activated screen of a display, thereby causing a semi- 
transparent window to be opened) (Partanen: page 1, paragraph 0016). However, Partanen and 
Skinner fail to explicitly teach accessing a database to obtain information. Gilai teaches a 
method comprising handwriting recognition (Gilai: column 23, lines 7-10) similar to that of 
Partanen and Skinner. In addition, Gilai further teaches accessing a database to obtain 
information (Gilai: column 5, line 63 - column 6, line 3). It would have been obvious to one of 
ordinary skill in the art, having the teachings of Partanen, Skinner and Gilai before him at the 
time the invention was made, to modify the method of determining whether a field is configured 
to receive text input of Partanen and Skinner to include the access of a database to obtain 
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information taught by Gilai. One would have been motivated to make such a combination in 
order to provide an organized and efficient way of indexing, storing and retrieving a large 
amount of information. 

10, Claim 24 is rejected under 35 U.S.C. 103(a) as being unpatentable over Partanen et al. 
U.S. Publication 2003/0001899 (hereinafter "Partanen"), Skinner U.S. Patent 6,661,920, as 
applied to claim 18 above, and Microsoft® Excel, copyright 1999 (hereinafter "Excel"). 

Referring to claim 24, Partanen and Skinner teach all of the limitations as applied to 
claim 18 above. However, although Partanen and Skinner teach increasing the size of the visible 
input window (re-sizing the displayed semi-transparent window) (Partanen: page 2, paragraphs 
0027-0031), Partanen and Skinner fail to explicitly teach increasing the size of the visible input 
window based on the data approaching an end thereof and to enable entry of additional data. 
Excel teaches an input interface that adjusts the appearance of the visible input window (see 
screenshots 2 and 3 attached at the end of the office action) similar to that of Partanen and 
Skinner. In addition, Excel further teaches increasing the size of the visible input window based 
on the data approaching an end thereof and to enable entry of additional data (see screenshots 2 
and 3). It would have been obvious to one of ordinary skill in the art, having the teachings of 
Partanen, Skinner and Excel at the time the invention was made, to modify the input interface of 
Partanen and Skinner to include increasing the size of the input window as needed, taught by 
Excel. One would have been motivated to make such a combination in order to allow users to 
enter as much information as needed, making it easier for them to input and view information. 
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11. Claims 30-33 are rejected under 35 U.S.C. 103(a) as being unpatentable over Partanen et 
al. U.S. Publication 2003/0001899 (hereinafter "Partanen"), Skinner U.S. Patent 6,661,920 and 
Frink et al. U.S. Patent 5,956,423 (hereinafter "Frink"). 

Referring to claim 30, Partanen teaches a system comprising an application program 
having at least one application input area into which user input data can be entered (the email 
program shown in Figure 1 has a plurality of user input areas such as Attachments, Subject, etc., 
into which user data can be entered) (Partanen: page 1, paragraphs 0014-0016); user interface 
code external to the application program (semi-transparent interface window for receiving hand- 
written input) (Partanen: pages 1-2, paragraph 0022, pages 2-3, paragraphs 0032-0040 and 
further shown in Figure 1); a typing engine that determines whether to call the user interface 
code for a selected application input area of the application program based on attribute 
information associated with that application input area (based on attribute information for the 
application input area, such as whether user has initiated manuscript input to a point of the input 
area on a touch-activated screen, it is determined whether the user interface code should be 
called, i.e. whether a semi-transparent window should be displayed) (Partanen: page 1, paragraph 
0016, pages 1-2, paragraph 0022 and pages 2-3, paragraphs 0032-0040), the user interface code 
providing a semi-transparent input area based on the attribute information when called, the semi- 
transparent input area configured such that a user may still enter data via user input into the user 
interface code (upon user initiation of manuscript input at point 36 on the display screen of 
Figure 1, a semi-transparent window is displayed relative to the field, allowing users to input 
manuscript characters, for example, the letter "I", while the field is operable to receive input data 
such as user input of particular letters and/or numbers from a virtual keyboard) (Partanen: page 
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2, paragraph 0032 and page 3, paragraph 0038); a gesture engine, the gesture engine invoked to 
determine whether user input data directed to the semi-transparent input area is a gesture directed 
to the application program or information that should be recognized as text (deciphering whether 
user input is a gesture, such as user selection of virtual buttons and keys from a virtual keyboard, 
or user input of written manuscript characters that requires handwriting recognition) (Partanen: 
pages 2-3, paragraphs 0032-0040); and a handwriting recognition engine, the handwriting 
recognition engine configured to receive the information that the gesture engine has decided 
should be recognized as text (when users input handwritten character, the pattern of the 
handwritten is relayed to the handwriting recognition software) (Partanen: page 2, paragraph 
0025), the handwriting recognition engine responding by returning recognized text when 
provided with the information (for example, recognizing the handwritten "I" and outputting it as 
the character "I" in Figure 1) (Partanen: page 2, paragraph 0022). However, although Partanen 
teaches a virtual keyboard, Partanen fails to explicitly teach simultaneously receiving typed and 
handwritten user input. Skinner teaches an interface that allows users to enter handwritten data 
similar to that of Partanen. In addition, Skinner further teaches receiving typed user input and 
the program field being operable to simultaneously receive typed and handwritten user input (the 
handwriting and typed keyboard input systems are simultaneously active and capable of data 
entry) (Skinner: column 2, line 41-column 3, line 3). It would have been obvious to one of 
ordinary skill in the art, having the teachings of Partanen and Skinner before him at the time the 
invention was made, to modify the handwriting recognition method of Partanen to include the 
simultaneous handwriting and typed keyboard input capabilities of Skinner. One would have 
been motivated to make such a combination in order to provide a more flexible user data entry 
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system, giving the user an option to choose the more convenient input mechanism depending on 
the function to be executed without the hassle of having to switch between the input systems. 
However, although Partanen and Skinner teach a timing mechanism (displaying the handwritten 
input for a predetermined period of time after the input from the input device, upon which the 
pattern is completed and recognized by the recognition software) (Partanen: page 2, paragraph 
0024 and page 3, paragraphs 0033-0035), Partanen and Skinner fail to explicitly teach a timing 
mechanism configured to cause removal of the semi-transparent input area when no user 
interaction with the visible input area is detected for a period of time. Frink teaches an interface 
for inputting and recognizing handwritten data (Frink: column 2, lines 37-65) similar to that of 
Partanen and Skinner. In addition, Frink further teaches a timing mechanism configured to cause 
removal of the input when no user interaction with the input area is detected for a period of time 
(sending the handwritten data to the recognition engine, and therefore erasing the input, when the 
user stops writing for a period of time) (Frink: column 2, lines 44-63 and Figures 2A, 2B and 
2C). It would have been obvious to one of ordinary skill in the art, having the teachings of 
Partanen, Skinner and Frink before him at the time the invention was made, to modify the input 
recognition interface comprising the timing mechanism of Partanen and Skinner to include the 
timing mechanism taught by Frink. One would have been motivated to make such a combination 
in order to allow users to easily edit the documents, reducing the confusion of mixing up editing 
commands and data input by the user. Furthermore, it allows users to perform functions such as 
note taking faster and more efficiently; recognizing characters after a certain time has elapsed, 
representing the user has completed taking notes, is faster and more efficient than translating the 
written data character by character as the user is taking notes. 



Application/Control Number: 09/976, 188 Page 1 7 

Art Unit: 2173 

Referring to claim 31, Partanen, as modified, teach wherein the recognized text is 
received by the user interface code and made available to the application program (handwritten 
text characters are input through the semi-transparent window and recognized text are displayed 
on the email program shown in Figure 1) (Partanen: page 2, paragraphs 0022-0025 and 0032). 

Referring to claim 32, Partanen, as modified, teach wherein the application program 
displays the recognized text in the application input area (text characters are displayed on the 
semi-transparent window and on area 20 of Figure 1) (Partanen: page 2, paragraphs 0022-0025 
and 0032). 

Referring to claim 33, Partanen, as modified, teach a growth rulebase, the growth 
rulebase determining whether to alter an appearance of the semi-transparent input area in 
response to the information received therein (pre-selected preferences that may be established 
such as the display size of the semi-transparent window can be altered) (Partanen: page 2, 
paragraphs 0027-0031). 

Response to Arguments 
12. Applicant's arguments filed 3 May 2006 have been folly considered but they are not 
persuasive. Due to the claims not being folly disclosed, the examiner is not able to determine that 
the arguments overcome the previous rejection. For example, the applicant argues that Partanen 
and Skinner fail to teach typed text to be entered from a physical keyboard. However, due to the 
claims being indefinite and not enabling, interpretation of the claim language is difficult and the 
examiner is unable to determine that the applicant's arguments overcome the previous rejection. 
For prosecution purposes, the claim language, as best understood by the examiner, were 
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interpreted to the best of the examiner's knowledge as previously presented (i.e. the claims were 
interpreted without the newly added limitation of "from a physical keyboard"). 

13. Furthermore, on lines 18-20 of page 1 1 of the remarks filed on 3 May 2006, the applicant 
argued that "In this manner, a user may enter information to the field through simple typing or 
through handwriting that uses a recognition engine to provide data to the field as well". 
However, this argument of the user being able to enter typed or handwritten data is contradictory 
to the claimed language of simultaneously receiving typed input from a physical keyboard and 
handwritten data. 

14. The applicant also argues that the office action is employing impermissible hindsight 
reasoning in combining Partanen and Skinner. In response to applicant's argument that the 
examiner's conclusion of obviousness is based upon improper hindsight reasoning, it must be 
recognized that any judgment on obviousness is in a sense necessarily a reconstruction based 
upon hindsight reasoning. But so long as it takes into account only knowledge which was within 
the level of ordinary skill at the time the claimed invention was made, and does not include 
knowledge gleaned only from the applicant's disclosure, such a reconstruction is proper. See In 
re McLaughlin, 443 F.2d 1392, 170 USPQ 209 (CCPA 1971). Both Partanen and Skinner teach 
similar graphical user interfaces for allowing users to enter handwritten and typed input via 
keyboard. Skinner specifically mentions that allowing users to simultaneously enter input via a 
handwritten input system and typed input system (i.e. keyboard) provides the advantages of 
flexibility, giving users the option to choose the more convenient input mechanism depending on 
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the function to be executed without the hassle of having to switch between the input systems, in 
column 2, lines 41-53. 

15. The applicant argues that although Skinner includes a typing mode and handwriting 
mode, switching between modes for data entry is still required, and therefore, Skinner does not 
teach allowing a user to simultaneously enter typed data and handwritten data. The examiner 
respectfully disagrees. Skinner explicitly states that "An embodiment of the present invention 
provides a data entry system wherein a handwriting recognition mechanism and an on-screen 
virtual keyboard are simultaneously active and capable of data entry", in column 2, lines 44-47. 

16. With respect to claim 10, the applicant argues that Partanen simply teaches expanding an 
input field when double-tapped and does not show any capability of assessing window attributes 
or window class information. The examiner respectfully disagrees. Partanen teaches that 
window attributes such as whether the user has contacted the displayed window with the stylus, 
are evaluated in order to display a semi-transparent window; furthermore, when it is determined 
that the user has contacted the window, a signal is generated to the data processor to access semi- 
transparent window's attributes, or class information, i.e. the preferences established for opening 
the semi-transparent window, as recited on page 1, paragraph 0016 and page 2, paragraph 0032. 
Therefore, the examiner respectfully maintains that Partanen teaches assessing window attributes 
or window class information. 
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17. Lastly, the applicant argues that the combination of Partanen with the teachings of 
Skinner and Frink is made by employing impermissible hindsight. In response to applicant's 
argument that the examiner's conclusion of obviousness is based upon improper hindsight 
reasoning, it must be recognized that any judgment on obviousness is in a sense necessarily a 
reconstruction based upon hindsight reasoning. But so long as it takes into account only 
knowledge which was within the level of ordinary skill at the time the claimed invention was 
made, and does not include knowledge gleaned only from the applicant's disclosure, such a 
reconstruction is proper. See In re McLaughlin, 443 F.2d 1392, 170 USPQ 209 (CCPA 1971). 
Both Partanen and Skinner teach similar graphical user interfaces for allowing users to enter 
handwritten and typed input via keyboard. Skinner specifically mentions that allowing users to 
simultaneously enter input via a handwritten input system and typed input system (i.e. keyboard) 
provides the advantages of flexibility, giving users the option to choose the more convenient 
input mechanism depending on the function to be executed without the hassle of having to 
switch between the input systems, in column 2, lines 41-53. Frink also teaches a graphical user 
interface that recognizes handwritten input similar to that of Partanen and Skinner. Furthermore, 
by combining the input interfaces of Partanen and Skinner, with Frink' s teaching of removing 
the input interface when no user interaction is detected for a period of time, users would be 
provided with a clear and easy to understand interface, reducing the confusion of mixing up 
editing and data input commands by having a clustered screen display with unnecessary 
components. 



Conclusion 
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Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Ting Zhou whose telephone number is (571) 272-4058. The 
examiner can normally be reached on Monday - Friday 7:00 am - 4:30 pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, John Cabeca can be reached at (571) 272-4048. The fax phone number for the 
organization where this application or proceeding is assigned is (571) 273-8300. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 
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